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ABSTRACT 



The use of exact-lenath blanks in cloze tests of 
readinq comprehension was invest ioated. Subjects were indiqenous 
primary-school pupils in New Guinea for whom p nolish was a foreian 
lanauaqe. No significant difference was found between the mean scores 
of subjects cn cloze tests usinq blanks of a mitorm length ard the 
mean scores of subjects on cloze tests usinq blanks of the same 
lenqth as the deleted words, noth versions of clove tests wer° 
equally valid as measures of general readinq comprehension. The 
question of Irnqth of blank to use in cloze tests has practical as 
well as theoretical sionificance for teachers in constructina cloze 
tests when they wish to use photocopyinq facilities. The implication 
of thes° findings is that by the use of a photocopy procedure, such 
factors as size of print, illustrative material, and paae lavout rav 
be included in the estimate of the ease or difficulty of printed 
material as measured bv cloze tests. Tables and references are 
included. (Author/RU) 
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The purpose of this study was to investigate the use of exaot-length 
blanks in dose teats of reading comprehension. Subjects for the study 
were indigenous primary school pupils in Hew Guinea for whoa English was 
a foreign language* Ho eignifioant difference was found between the mean 
scores of subjeots on close tests using blanks of a unifora length and the 
mean scores of subjeote on dose tests using blanks of the same length as 
the deleted words* Both versions of doze tests were equally valid as 
measures of general reading comprehension • The implication of these 
findings is that such faotore as sise of print) illustrative material and 
page layout may be inoluded in the estimate of the ease or difficulty of 
printed material as measured by close tests* 
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The present paper is a report of research on clone procedure, a technique 
that researchers, reading specialists and classroom teachers are finding 
increasingly useful to measure the reading difficulty or readability of 
printed material* 

Cloze procedure has been described by Taylor (Vl^) v/ho introduced it as a 

psychological tool for measuring the effectiveness of communication# Basically, 

the procedure consists of a set of rules for constructing oloze tests over 

samples of written (or spoken) materials, administering these tests to subjects 

and scoring them, and determinirig from the cloze scores the degree of 

comprehension of the materials* The construction of a close teat simply 

requires words to be deleted through a passage according to scoe meohanical 

system* A subject is asked to replaos the missing words and the number of 

wo* Is he correctly replaces is an index of his comprehension of the passage. 

Thera is a sound thooretioal basis for cloze scores as measures of 

reading comprehension* Pries (j£) identified three layers of language 

meanings i the layer of meanings carried by the grammatical structures, the 

layer of meanings carried by the lexical items, and the layer of sooial* 

cultural meanings* Oloze procedure taps these three layers of language 

meanings for, as MoLeod (j) stated, to successfully reconstruct a message 

requires a familiarity with the grammatical etruoture of English, 
an understanding of lexical meaning and, if the passages 
selected are ooncemed with a variety of experiences familiar 
in a given culture, they refleot to some extent * jooial-cultunal' 
meaning* (2» p»3 #) 




The set of xvtles for constructing does tests is quite well 
established* Comprehensive reports by Rankin (§, 2) and by Anderson Q) 
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reviewed studies on close procedure, studies carried out in the main in 
English speaking parts of the world with native speakers of English# 

There is, however, a Growing body of research showing that cloze 
procedure may be more widely applied, for example, to English and to 
French learned as foreign languages 4), to Japanese (l 0 )> and to 
Korean (12). 

Previous studies on cloze procedure have invariably employed blanks 
of a standard length to denote words deleted from a pessage following 
Taylor's ( 1 1 ) assumption that length of blank influences "guessing"# 

Thus, Bormuth in his definition of oloze readability procedure states 
that "the deleted words ore replaced by underlined blank spaces of a 
unifora length . . » rt (2, p# 1 • ) The reasoning behind this assumption 
is presumably that the length of the blank ie a olue providing additional 
information about the deleted word# However, it ie difficult to predict 
the way in which this olue might operate# If a subject is very familiar 
with the layer of meanings carried by the grammatical structures, with 
the layer of meanings oarried by the lexical items, and with the layer of 
aooial-cultural meanings, this additional clue might be quite unnecessary* 
On the other hand, to the extent that a eubjeot is not familiar with 
the various layers of meanings in written language, the additional clue 
might prove of little assistance# Sheee considerations suggest the 
operation of at least two important variables! the reading difficulty 
of the written language and the comprehension ability of subjeots# 
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Purpose 

The purpose of the present study was to compare the use In cloze tests 
of standard or uniform length blanks with blanks of the same length as the 
word deleted* Comparisons were made U 3 ing (a) passages varying in 
difficulty and (b) subjects differing in reading ability* The null 
hypotheses tested were: 

1 • There is no difference between the mean score for standard- 
length blanks and the mean score for exact-length blanks at 
different levels of passage difficulty and for different 
reading ability levels* 

2. There are no interactions between length of blank, difficulty 
levels and reading ability levels* 

The five per cent level of signifioanoe was required for the rejection 
of the hypothesee* 

This study was part of a larger study conducted in Hew Guinea. The 
Hew Guinea setting provided an opportunity to test cloze procedure as a 
measuring instrument with subjects learning English as a foreign language* 
Further, it provided an opportunity to test cl 02 e procedure against a 
multilinguistio background of great diversity and perplexity for it has 
been estimated that over five hundred distinot languages are spoken in 
Hew Guinea* 

Test Measures 

Cloze tests were constructed over passages drawn from three children's 
readers* The three readers were judged difficult, neither difficult nor 
easy, and ea6y reading for children in the upper primary levels of 
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Territory schools* There was a lead-in of ten words to each passage and 
thereafter every eighth word .vas systematically deleted through the passage# 
In half the cloze tests missing words were indicated by underlined blanks 
ten spaces in length (standard-length blanks)} in the other half, blanks 
were the same length as the word deleted (exact-length blanks)# Since the 
exact-length versions of the oloze tests did not allow sufficient space for 
subjects to print their answers, answer spaces were provided alongside the 
line in which the deletion appeared# The same procedure was adopted for 
the standard-length versions so that only length of blank varied# 

For example, 



Standard-length version 

When the tiger is too old to hunt for 
food the jungle, 

Exact-length version 

When the tiger is too old to hunt for 
food ^ the jungle, 



ANSWERS 



To control for initial reading ability, eubjeots were first ranked 
according to their scores on Watts' Test (13)» This test was shown by 
KeLeod ( 2 ) to correlate highly with oloze tests* 



Subjects 

All testing was carried out in one primary echool. The predominant 
local language in the village where the echool was situated was Pidgin 
while in the school the nedivn of instruction wae English# All pupils in 
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the top three standards (N=110) constituted the sample* 

Experimental Design 

An experimental design that allows the effect of length of blank to 
be tested at different levels of passage difficulty and for different 
levels of reading ability is tliree-diraensional analysis of variance# The 
advantage of matching subjects on a control variable (the Watts' Test) is 
that an important source of error is reduced thereby increasing the preoision 
of the experiment. 

The design called for two testing sessions* In the first, the Watts' 
Teat was administered to each standard* Cn the basis of ’Watts' scores, 
an order of merit was made for the 110 subjects. High, average, and low 
reading ability groups were formed by dividing pupils into top quarter, 
middle half, and bottom quarter respectively* Within each ability group 
subjects were randomly allotted to standard- or exaot-length versions of 
close tests at one of the three levels of passage difficulty* This 
reduced N to 96* To satisfy the condition of proportionality from row 
to row, high and low ability treatment cells each contained four subjects 
with eight subjects in each average ability treatment oell. Three days 
later the criterion dose tests (standard- And erauVlength) were 
administered under untimed conditions* 

To minlmuse what Lindquist (6) calls Type 0 (or group) errors, 
"treatments" were administered at the same time in olass groups* Each 
treatnen": group was made up of approximately proportional mnbers of 
pupils frcn each standard and order of class testing wps radcoi^ed* 
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A further comparison between standard- and exact-length blanks was made 
by correlating scores on eacn of these forms of cloze tests with scores on 
the Watts' Test for each level of difficulty. This, in effeot, gave an 
estimate of the concurrent validity of cloze tests as measures of general 
reading comprehension. 

Results 

The key assumptions underlying analysis of variance are homogeneity 
of variance and noiraality of distribution. The former v/as tested for the 
main comparison groups (length of blank and passage difficulties) using 
P test for differences between variances of independent samples and 
Hartley's P jnax test (j>) respectively; and the latter was tested by 
inspection. There was no evidence to suggest either assumption was 
unreasonable. 

Lindquist's (j6) procedure for testing main and simple effects >vas 
followed. The summary of the analysis of variance is presented in Table 1. 
Keans, standard deviations, and the nisnber of subjects on which these were 
based are ehown in Table 2. 



Tables 1 and 2 here 

Inspection of Table 1 shows that no interaction terra was significant* 
>!or was there a significant difference between the two versions of cloEe 
tests. Thus there was no basis for rejecting either Hypothesis 1 or 
Hypothesis 2. 
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The differences between passages was highly significant which was to be 
expeoted since this was the basis for selecting the passages. The purpose 
of dividing subjects into reading ability levels was to inorease the 
precision of the experiment by matching subjects on a related variable# 

The product moment correlation coefficients for each group of 16 
subjects on the Watts' Test and the standard- and exaot-length versions of 
cloze tests for eaoh of the three passages appear in Table 3» All 
correlation coefficients were significant at the five per cent level# 

A single estimate of the correlation for each type of blank and the Watts' 
Test was made using Fisher's z test# The mean z value for the three 
passages was calculated and transformed to the corresponding r# The last 
row of Table 3 shows this average coefficient of correlation# 

Discussion and Conclusions 

The principal finding in this study concerned differences between 
the mean score for standard-length blanks and the mean score for exaot- 
length blanks# Although the latter yielded slightly higher scores than 
the former for passages at three levels of difficulty, this difference was 
not significant. For the subjects and passages used in the present study 
it mattered little whether blanks were of a uniform length or the same 
length as the deleted word# 

Signifiean. correlations between the two versions of cloze tests and 
the Watts' Test showed that the vise of exact-length blanks was no lees 
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valid a procedure for measuring general reading comprehension than the 
use of standard-length blanks* 

The question of length of blank to use in cloze tests has practical 
as well as theoretical significance* ltor example, if a teacher wishes to 
construct a cloze test for use with his pupils or if a reading centre 
wishes to calibrate a number of books, the usual procedure is to type the 
passage deleting words systematically through it (for example, every 
fifth or eighth word) and indicate deletions by lines of uniform length. 
However, with the general availability of photo-copying facilities it is 
simpler and more convenient to blank out words for deletion, provide a 
space for recording answers at the right hand side of the page, and run off 
multiple copies* This process necessitates having blanks of the same length 
as the word to be deleted* 

The advantage of the procedure suggested above is that it allows suoh 
faotors as size of print, illustrative material, and page layout to be 
incorporated in, not excluded from, the doze estimate of readability* 

If the findings of this study are confirmed by other studies, cloze 
procedure may prove an even more powerful technique for measuring 
readability* 
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TABLE 1 



Three-Dimensional Analysis of Variance of Criterion 
Scores of 96 Subjects Classified by Beading Ability 
Level within Passage Difficulty Iovels and Length 
of Blank in Oloze Teste 



Source of Variation 


df 


SS 


MS 


P 


P 


Blanks (A) 


1 


555.844 


555.844 


2.629 


NS 


Passages (b) 


2 


26855.146 


13427.573 


63.498 


o 

o 

• 

V 


Levels (l) 


2 


23472.282 


11736.141 






Interaction (AxB) 


2 


273.938 


136.969 


.492 


NS 


Interact! on (AxL) 


2 


35.113 ' 


'""17.557 


.061 


NS 


Interaction (BxL) 


4 


2865.062 


716.266 


2.498 


NS 


Interaction (AxBxL) 


4 


845.855 


211.464 


.738 


NS 


(Cells) 


(17) 


(54903.240) 








Within (Y/) 


78 


22361 .750 


286.689 






Total 


95 


77264.990 










TABLE 2 



Sample Means and Standard Deviations for three Passage 
Difficulties under Standard- and Exaot-hength Blanks 



Passage 

Statistics 


Standard-Length 






Exact-Length 




A 


B 


C 


Total 


A 


B 


C 


Total 


Mean 


58.19 


40.13 


15.44 


37.92 


63.38 


40.63 


24.19 


42.73 


Standard 

Deviation 


24.99 


26.15 


12.86 


28.25 


17.17 


31.87 


17.72 


28,28 


N 


16 


16 


16 


48 


16 


16 


16 


48 




TABLE 3 

Validity Coefficients at Three Levels of Diffioulty 



Difficulty Levels 


Watts and Standard-Length 


Watts and Exaet-Length 


Passage A 


.51* 


.65 


Passage B 


.92 


.73 


Passage C 


.75 


.83 


M r 


.61 


.75 



* A correlation coefficient of 0.426 is required for significance 
at the 0.05 level with 14df. 
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